Towards the Automatic Construction of Conceptual Taxonomies

In this paper we investigate the possibility of an automatic construction of conceptual taxonomies and evaluate the achievable results. In our meaning, a concept is represented by a keyword contained and extracted from a text corpus. A conceptual taxonomy is then a hierarchical organization of the keywords (Keyword Hierarchy, KH) such that the keywords at the higher hierarchy levels are representatives of a higher number of other keywords and as a consequence there is a higher number of documents that contain them. The hierarchy construction, on the space of the keywords, is performed by Ward hierarchical clustering algorithm, guided by a keyword proximity measure that is wellknown in statistics for the association between two categories: Goodman-Kruskal τ . Then, we perform a choice of a keyword representative from each cluster of keywords in the hierarchy. This is done in a similar way in which PageRank determines the authority of Web pages. The obtained hierarchy has the same, several advantages both descriptive and operative of indices on keywords which perform a partitioning of a large document collection with respect to the search space of the contents. In addition, we have a description of each cluster in the hierarchy by a list of frequent occurring terms ordered by their authority score determined by PageRank. We performed experiments in a real case the abstracts of the papers published in the last 8 years in ACM Transactions on Database Systems Journal in which the papers have been manually classified into the ACM Computing Taxonomy (CT) whose categories were created by computer science experts. The obtained keyword hierarchy provides interesting insights on the documents content and constitutes a useful browsing tool on the papers. We evaluated also objectively the generated hierarchy. We evaluated the correspondence between the clusters in KH and the categories in CT both by Jaccard measure on the documents and by entropy in terms of the category of documents in the clusters and obtained good results. We also evaluated the capability of classifiers to classify in the categories of the two taxonomies showing that KH provides a greater facility than CT.